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articipatory approaches to evaluation are rooted in the idea that 
the people who are most affected by the research should be 
partners in designing it and carrying it out. These approaches 
focus on topics that are meaningful to the affected individuals 
and on research that is carried out in ways that are sensitive to their expe- 
riences. Intentionally engaging affected groups in the research process can 


lead to new insights about the topics being studied. 


MDERC researchers and a group of young people used participatory evaluation approaches 
to understand the experiences of young people as they engaged in program services and 
worked toward their goals. This participatory work was completed as part of a research 
project undertaken for the benefit of the Casey Foundation. The research findings high- 
lighted the ways that young people describe and work toward their employment and edu- 
cation goals. It also documented the critical value of supportive adult relationships in help- 
ing young people persist in working toward goals that can take multiple years to achieve, 
as well as the need for additional resources for programs to support young people with 
housing and mental health. The young people who took part in this initiative were partic- 
ipating in a fellowship opportunity (“LEAP Youth Fellows”) available through Learn and 
Earn to Achieve Potential (LEAP)™. LEAP is a multisite initiative of the Annie E. Casey 
Foundation that aims to help youth and young adults ages 14-25 who have been involved 
in child welfare or justice systems, or experienced homelessness, succeed in school and at 
work by building and expanding their education and employment pathways. This brief 
describes the benefits of the collaboration to the research process, strategies used by the 
researchers to structure the collaboration, and the challenges they encountered. Other 
researchers can use this example to inform their own efforts to incorporate participatory 
methods into their research to advance equitable evaluation practices, and ultimately, to 


improve an evaluation by incorporating the knowledge and voice of those with lived expertise in the 
systems and programs researchers study. Box 1 provides more information about the LEAP initiative 
and grantees. 


BOX 1 
ABOUT THE LEAP PROGRAM 


Grantees taking part in LEAP implement education- and employment-focused program models to 
help young people ages 14 to 24, advance along educational and employment pathways. One model, 
Jobs for America’s Graduates, or JAG, targets young people who have not completed high school. 
The program seeks to help these students obtain a high school credential and to equip them with 
the work and life skills they need to access quality jobs or acquire a postsecondary education. The 
second model, Jobs For the Future’s Back on Track, helps young people transition to postsecondary 
education and persist through their first year of college or advanced training. Young people in LEAP 
have current or prior child welfare or justice system involvement, or prior experience with homeless- 
ness. They face a set of systemic and structural barriers, which can hinder young people’s progress 
in programs designed to elevate their educational and economic opportunities. 


LEAP grantees typically operate the program in multiple locations and in partnership with other or- 
ganizations, such as the K-12 educational system, postsecondary education and training institutions, 
employers, workforce development organizations, child welfare and justice agencies, and other local 
nonprofit organizations and government entities. They operate the JAG and Back on Track as core 
models, but also provide additional support and adapt practices informed by research about what else 
might benefit this population of young people. These include training LEAP staff in trauma-informed 
approaches that help staff members to recognize the symptoms of trauma and to understand its effects 
on behavior; developing partnerships and instituting practice changes within systems, including the 
entities mentioned above, to help young people access support and mitigate barriers; and offering 
young people leadership opportunities in the program. To learn more about LEAP, see MDRC’s prior 
study of the program’s implementation, outcomes, and costs. 


ABOUT PARTICIPATORY EVALUATION 


Participatory research is an “orientation to inquiry,” or a way of thinking about and relating to 
research that challenges many of the assumptions central to traditional research.” Below are some 
key principles behind the approach.* 


e People do not need advanced degrees to conduct research. Everyone can bring valuable 
insights to all parts of the research process, from designing the research to interpreting 
and sharing the findings. 
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e Those who are closest to a problem generally know the most about it. People’s sub- 
jective and lived experiences bring valuable insights and contributions to the research 
process; these perspectives are as valuable as the “objective” viewpoint of outsiders. 


e ‘The inclusion of a broad range of perspectives will produce better, more accurate 
insights. Bringing together people with lived experience, community members, and 
researchers provides opportunities to collect data in ways that are culturally responsive 
and to interpret the data from many perspectives, making them less subject to the biases 
of a single perspective. 


USING PARTICIPATORY METHODS FOR INTERVIEWS WITH LEAP 
PARTICIPANTS 


The National LEAP Youth Fellowship engages one young person from each LEAP program loca- 

tion to inform LEAP’s youth engagement and leadership development efforts. To be eligible for the 
fellowship, young people must be between the ages of 18-25; enrolled in LEAP at the start of the 
Fellowship; and interested in developing leadership skills, supporting their LEAP program and peers, 
and improving conditions in their communities for young people who may have similar experiences. 
The fellowship is a one-year commitment, and fellows received honoraria for their participation. Each 
fellow develops an individualized learning plan to work toward professional goals, explore interests, 
and develop skills. The program is coordinated by the Annie E. Casey Foundation (Casey) and the 
School and Main Institute (SMI) in coordination with LEAP programs. 


The MDRC team, which consisted of the authors, who were working on the overall evaluation, and 
the fellows worked together through seven virtual meetings held by Casey and SMI. In workshops 
designed and led by MDRC, the fellows and the MDRC team worked together to develop guides for 
interviews with LEAP participants that explored what can make it easier or harder for young people 
to take part in LEAP and work toward their goals. MDRC used these interview guides to conduct 
interviews with LEAP participants. Later, the fellows helped interpret a specific portion of the inter- 
view data and generate lessons for the program going forward. The fellows also reviewed the lessons 
from the research and suggested the best ways to present their collaborative findings. In addition to 
honing the presentation for LEAP, fellows learned research and interview skills, a secondary goal 
of the participatory research initiative. Box 2 provides an overview of the activities MDRC and the 
fellows took part in during this research. 


Each participatory endeavor is different in terms of what roles people play and how they collabo- 
rate on the research. In this work, MDRC and the foundation developed the overarching research 
questions based on MDRC’s prior work with LEAP. The fellows provided input on what should be 
explored or covered within these questions. Then, MDRC staff members generally took the first step 
at drafting materials, such as interview guides and this brief, and then revised them with the fellows’ 
input. Time and budget constraints meant fellows were not able conduct interviews, interpret all 
parts of the data, or write up the findings. Though the engagement was limited in scope, from the 
researchers’ perspective, the collaboration greatly strengthened the research process and what was 
learned from the project. These results are described below. 
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BOX 2 
PARTICIPATORY RESEARCH IN LEAP 


Develop a shared understanding of research practices and the previous LEAP study findings. 
« Fellows and MDRC learned about evaluations and participatory research 
« Fellows and MDRC discussed findings from previous research on LEAP 
« Fellows and MDRC get familiar with goals for current evaluation 
Develop the interview guide. 
* Fellows brainstormed topics to include in the interview guide 
« Fellows tested the interview guide by conducting mock interviews 
* Fellows refined the interview guide, promoting additions about racial justice and the pandemic 
Conduct the interviews. 
« _MDRC conducted interviews 
Analyze the data. 
* _MDRC coded the data 
« Fellows analyzed selected excerpts about motivation, a topic they included in the interviews 
« Fellows developed recommendations for LEAP programs based on the data 
Share the findings. 
« _MDRC summarized the findings from the research 


« Fellows reviewed and shaped the lessons coming out of the participatory research work 


The project also responded to the events that unfolded during 2020, including the COVID-19 pan- 
demic and a broad, national movement for racial justice and policing reforms. Events necessitated a 
shift from a shorter in-person research development process to a longer virtual one. MDRC and the 
LEAP fellows expanded the breadth and detail of the questions and topics covered in interviews, and 
incorporated pauses in developing protocols to spend time reflecting and building up the fellowship’s 
virtual community. 


RESULTS FROM LEAP’S PARTICIPATORY APPROACH 


Result 1: A More Perceptive Interview Guide 


Bringing in the fellows led to an interview protocol aligned more closely with the experiences of 
young people in LEAP and so yielded better data. The research team was prompted by the fellows to 
ask about topics that MDRC would have overlooked, including motivation and the role of mentors 
in and outside of LEAP. The fellows especially emphasized the role that mental health can play in a 
young person’s life, encouraged the research team to ask about these topics, and helped ensure the 
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questions were framed in ways that were respectful and approachable. These topics were central to 
the research findings about persistence that resulted from this work. Figure 1 highlights some ques- 
tions that were added to the interview protocol because of the fellows’ suggestions. 


FIGURE 1 


INTERVIEWING GUIDE THEMES DEVELOPED BY LEAP FELLOWS 


BRAINSTORM THEMES INTERVIEW QUESTIONS 


fe ee ee ; 1. Has something with your mental health—like anxiety or 
Mental health makes a depression—ever made it hard to take part in a program, 
| big difference in feeling | school, or work? 

| motivated and getting | 2. Have you gotten any help with your mental health? What 
did it look like? 

3. What kind of help would you have wanted, ideally? 

I ] 


the program, school, 
or work. 


NERS eee oe ee ~ 1. What helps you work towards your goals? 
2. Has there ever been a time you lost motivation to work 
towards your goals? What was going on at the time? 


I 
] 
Personal motivation | 
How did you regain your motivation? Who or what helped 
I 
I 
I 
] 
I 


plays a central role in 


working towards your you? What did they do? 


goals. 3. In your experience, what role do you think personal 
motivation plays in working towards your goals? 
wePS sees ee ea ee “1. In your words, how do you define what a mentor is? Do 


you have someone you consider to be a mentor? This 
person can be a LEAP staff person or someone else. 
How did this person become your mentor? 

2. Has this mentor played a role in your choice to take part 


I I 

I I 

I I 

I I 

I I 

1 Young people want | 

mentorship and ad- ; 

| vice as they work to- | in LEAP? 
| wards their goals. 1 3. How does this person support you in working towards 
your goals? What does this person do if you make a 
mistake or have a setback in working towards your goals? 
! 1 4. Why do you think it’s helpful to have a mentor? 


Result 2: Increased reflection and intentionality in the interview process 


In addition, developing the interview guide collaboratively, with repeated rounds of input from 
the fellows, helped the MDRC team reflect on the interview process and how it would be received 
by LEAP interviewees. Preparing for the workshops with the fellows prompted MDRC to clarify 
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whether—and why—to ask various questions and how to get the most meaningful responses from 
the young people they were interviewing. 


Result 3: Improved understanding of research processes and applications 


Fellows said taking part in participatory research gave them a better understanding of the role 
research plays in shaping policy and practices. The project also highlighted applied and practical 
qualities of rigorous research by showing the fellows how it can be used to shape and improve every- 
day life. Fellows said they would have liked to learn more about research methods, build up more 
technical skills, and take part in more aspects of the research. The fellows are recognized in Box 3. 


BOX 3 
LEAP FELLOWS 


The LEAP evaluation addendum benefited greatly from partnering with the LEAP fellows in this re- 
search. MDRC is grateful to the fellows for their insights and engagement in this work: 

¢ Alicia Colon, NMCAN, Albuquerque 

¢ Hayley Brooks, University of Southern Maine, Lewiston 

- Alexis Valencia, SBCS, San Diego 

* Lakeshia Colbert, Nebraska Children and Families Foundation, Omaha 

¢ Lamont Harrell, Coalition for Responsible Community Development, Los Angeles 

¢ Elieser Nieves, Center for Alternative Sentencing and Employment Services, New York 

« Prince Riggins, The Door, New York 

« Ruby Jensen, Project for Pride in Living, Minneapolis 

* Teresa Davis, Jobs for Arizona’s Graduates, Peoria 


* Trey Greenewald, Covenant House Alaska, Anchorage 


LOOKING AHEAD 


This work offers several lessons for future participatory research activities with young adults. 


Design activities to make the most of young people’s limited time. The degree to which fellows 
participated in meetings and research activities varied. Some activities that asked fellows to take 

on research tasks outside of scheduled meeting times had fairly limited engagement, such as testing 
the interview protocol. As this research underscores, young people in LEAP have many pressures 

on their time that can make it challenging to accomplish everything they want and must do. This 
challenge felt even more acute during the pandemic. (Of course, it’s also possible that MDRC did not 
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present information in a way that was engaging or approachable, which could also have a negative 
effect on engagement.) Participatory research activities may be most successful when they maximize 
what can be done in the time young people have already set aside for the endeavor, such as reoccurring 
meetings. Strategies that allow for various points of entry (and exit), such as breaking the research 
process into small chunks or activities, may also align with young people’s availability and facilitate 
their participation. 


Weigh the pros and cons of working within an existing group. The LEAP Youth Fellowship was not 
specific to this research endeavor, and fellows worked on a wide range of initiatives, such as self-led 
“mini-grants” sponsored by Casey, leading community dialogues, and contributing to conversation 
about how to improve LEAP. Youth-serving organizations that engage in participatory research may 
benefit from weighing the pros and cons of working within existing groups like the LEAP fellowship, 
such as classes, cohorts, or leadership groups. Building participatory research activities into existing 
programs has many advantages in terms of accessing and coordinating stakeholders. In this case, it 
also provided an opportunity to expose a greater number of young people to research and its effects 
on programs and policies. However, working with an existing group may mean that the work is not 
fully aligned with young people’s interests. It’s possible that some fellows may not have been inter- 
ested in research overall or the particular topics that were being studied. Researchers and programs 
can address this by recruiting young people specifically to take part in a research engagement and by 
engaging young people in choosing the topic of study and research questions. 


Bring young people into more parts of the research process. MDRC led several aspects of data 
collection and analysis for this research due to budget and time constraints. However, young people 
frequently pointed out that who asks the interview questions matters just as much as the questions 
themselves, and that the identity of the interviewer can affect the quality and honesty of information 
shared during an interview. Likewise, in the analysis workshop, fellows saw things in the data that 
the MDRC team missed. While it may be labor intensive, creating additional opportunities for youth 
fellows to participate in data collection and analysis would likely increase the richness of the data col- 
lected and its interpretation. 


Continue to navigate privacy concerns. LEAP youth fellows are both a part of their LEAP commu- 
nities and part of the research team. Their positioning as “participant-researchers” was a strength 
that brought valuable insights to the research process, but it also raises questions about how to ensure 
the privacy of individuals who take part in interviews. 


This is important to research because when researchers collect data, they disclose who will see the 
information and how it will be used. That helps the people taking part in the research make informed 
decisions about whether they want to share their information with the research team. Given this con- 
sideration, it is important to limit the number of people who will see information to those who need to 
see individual responses, and to keep the information shared private so individuals cannot be identified 
in the research findings. 


This promise of confidentiality can make interviewees more comfortable sharing information, and 
it can also protect them from retaliation or stigma associated with information disclosed in the 


Strengthening Research Through Participatory Methods 7 


interview. One strategy to help mitigate concerns around privacy is to have groups establish robust 
community agreements about privacy and confidentiality. In addition, not all research questions 

or methods have the same risks. Professional researchers or adult advisors can help young people 
understand which questions or data collection methods may be more sensitive and help young people 
make informed decisions about whether and how to collect information in ways that will protect 
research subjects. While this research relied on interviews, other data collection methods may pose 
fewer risks, such as having fellows share their own stories through photo and text galleries known 

as “PhotoVoice,” video testimonials, or mapping exercises where young people draw connections 
between their communities, social or institutional networks, or relationships.* 


Build trust. Because it is inherently collaborative, participatory research requires trust and openness. 
Because the fellowship began at the start of the pandemic in a virtual environment, LEAP fellows 

did not know each other or the MDRC researchers well at the start of this project. This may have 
dampened their participation in some activities. Providing multiple ways to engage in activities, such 
as mixing discussion with text-based chats, increased the number of young people who shared their 
perspectives in workshops. Future participatory endeavors would also benefit from a longer start-up 
period that builds in time for developing rapport. 


CONCLUSION 


This participatory project is part of MDRC’s journey toward improving how its research honors the 
direct, lived experiences of people who participate in this research as experts in the systems and pro- 
grams being studied. The credibility of MDRC’s research rests on looking at problems from multiple 
perspectives to maximize its relevance and accuracy. Toward this effort, MDRC’s Equity Collabo- 
rative was formed in 2018 to infuse thoughtful and systematic application of culturally responsive, 
equity-based, and inclusive practices across MDRC’s research and technical assistance projects. The 
lessons about participatory research from this study will contribute to MDRC’s growing knowledge 
of the benefits, challenges, and potential solutions to working alongside community members to con- 
duct research and allow others to learn from and build on these lessons. 
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